Joint Education Initiative/Joint Education Initiative.iso/imagery/comets/halley/imaguide.txt

Text File | 1990-06-25 | 25KB | 317 lines

***** File IMAGUIDE.TXT



                     Comet Halley Archive Image Data
           User's Guide to the Compact Disc Read Only Memory



                               Contents

1. Background and Scope of Compact Disc
2. Accessing the Disc
3. Data Products
4. Supplementary Directories
5. Filenaming Convention
6. Available Software
7. References



1. BACKGROUND AND SCOPE OF COMPACT DISC

     This compact disc contains wide-field images of comet Halley obtained by
the Large-Scale Phenomena Network (LSPN) of the International Halley Watch
(IHW) during the interval of the Halley Armada spacecraft encounters:  1986
March 06-14.  The images were archived, digitized, and prepared for deposit on
CD-ROM by the Large-Scale Phenomena (L-SP) Discipline Specialist Team at the
NASA-Goddard Space Flight Center, Greenbelt, MD.  This disc contains 70
selected images from "Halley Armada Week", and 10 associated images of
various calibration objects; the data have been compressed to allow more data
stoarge on the disc.  The image selection criteria were, in the main, quality
of individual images and contribution to the overall temporal coverage of
Halley Armada Week.  A secondary criterion was the desire to maximize the
number of represented observatories.

     The total number of L-SP digitized images for the entire 1985-86
apparition exceeds 1600, which, combined with the typically large size of each
datafile, has resulted in a substantial set of CD-ROMs being required for
storage of the IHW/LSPN data alone, even in compressed form.  Because of this
disparity in the volume of data between L-SP and the eight other IHW
disciplines, it was decided to deposit the wide-field images on a series of
dedicated L-SP CD-ROMs; this disc is a special Halley Armada Week supplement
to that series of compressed L-SP data.

     The L-SP compressed image CD-ROMs are, of course, part of a larger set
which also includes data from the eight other IHW disciplines:  Amateur,
Astrometry, Infrared Studies, Meteor Studies, Near-Nucleus Studies, Photometry
and Polarimetry, Radio Studies, and Spectroscopy and Spectrophotometry.  The
data from these disciplines, obtained from 1982 through 1989, have been
placed on the same CD-ROMs, interleaved together in chronological sequence.
These multi-discipline CD-ROMs are sometimes referred to as the "mixed discs",
on which also reside subsampled, uncompressed L-SP "browse images"; the
rationale was that some representation of ALL the disciplines' data should be
in one place.  In addition to groundbased data returned from the nine IHW
disciplines, spacecraft data from the so-called "Halley Armada" have been
collected and placed in the Comet Halley Archive.

     Much more on the history of the IHW and on the specific operational
details of each discipline is given in the appendix and other text files on
the mixed discs, and this includes the Large-Scale Phenomena Discipline.
However, additional details on L-SP beyond those brief ones mentioned here are
also to be found in the various text files in the DOCUMENT directory of these
compressed image discs, especially NETLARGE.TXT and ACKNWLDG.TXT.


2. ACCESSING THE DISC

     These discs have been manufactured following the normal specifications
for CD-ROMs as well as the logical structure outlined by the ISO 9660
document.  A basic system requirement demands that the CD-ROM reader and
software driver support this standard.  At this writing (July 1990), most
access to the disc will be using IBM-PC or compatible computers.  However,
software support from vendors for the Apple MacIntosh II and some workstations
(SUN, MicroVAX) is now available.

     Assuming that you are using a PC style machine, the reader that you
bought came with the appropriate hardware (usually PC-bus controller board and
cable) and software to address your device.  Software packages should include
a device driver for your machine and extensions to DOS that allow the reader
to "look" like another random access disk drive; the de facto standard here is
the Microsoft Extensions (MSCDEX 2.0 or higher).  Install the specific device
driver software as directed by the documentation accompanying the hardware.
Usually, this will involve specifying the device driver in a DEVICE= line in
the CONFIG.SYS file and invoking the extensions package from your AUTOEXEC.BAT
file.  This will automatically configure the CD-ROM reader when the computer
is booted.

     Remember that the CD-ROM reader is assigned a drive name which you have
either chosen with the appropriate command line switch (/L:H assigns the CD-
ROM reader to drive H:) or allowed to default to the next available device
letter in your system.  Typical PC configurations will have two floppies (A:
and B:), and a hard disk (C:).  Put your drive at some letter higher than
that.  Remember that many DOS commands work on the CD-ROM but that it is a
read-only medium.  Some of the useful DOS functions are:

        DIR   - directory listing
        CHDIR - change directory (also CD)
        TYPE  - list the contents of a file on the screen; useful
                for normal text with <carriage return>, <line feed>
                delimiters
        COPY  - copy file(s) to another device
        PRINT - print file on hard copy device

     Notice that commands such as DEL and MKDIR are not available since the
CD-ROM is a read-only medium.  Furthermore, some compact disc software
packages will invoke screen plots that may depend on the DOS program
GRAPHICS.COM.  In this case, execute it in your AUTOEXEC.BAT file or before
you run the software package.

     The IHW CD-ROMs have been constructed to allow maximum access to the data
using some existing software developed within NASA, as well as user-supplied
programs.  In particular, each data file was originally supplied in the FITS
format, and also has a detached PDS label to describe the data structure.
Furthermore, the normal FITS files have had the headers placed in separate
files (with extension .HDR) from the data (with extension .IBG, or .IMQ).
The headers have been separated to allow unrestricted access to the data by
non-FITS programs.  The sizes of all header and data files have been preserved
as integer multiples of 2880 bytes (required by FITS) in order to facilitate
the reconstruction of the original FITS bytestream by concatenating the data
file onto the appropriate header file.


3.  DATA PRODUCTS

     Hundreds of scientists worldwide contributed to the success of the IHW
Archive generally, and to the Large-Scale Phenomena Network (LSPN)
specifically.  The participating LSPN observatories, with their IHW system
codes, instrument descriptions, and observers, are listed in the file
LSPNOBS.TXT in the DOCUMENT directory of this CD-ROM, and are indexed in the
file LSPNOBS.IDX of the INDEX directory.  The same and additional information
about the observations are contained in the individual FITS header files.
Details of the L-SP Discipline and Network activities, the types and extent of
data processing, etc., are given in various of the text files (especially
NETLARGE.TXT and CALIB.TXT) in the DOCUMENT directory of this CD-ROM, as well
as in the L-SP Discipline appendix on the "mixed discs".

     In an effort to reduce the space requirements for the Halley Archive,
data compression was applied to the Large-Scale Phenomena images using an
algorithm which converts 2-byte pixels to 1-byte pixel-to-pixel differences
whenever possible.  The result, essentially, is a coded byte stream of 8-bit
data.  This "previous pixel" technique was used for images of the comet
Giacobini-Zinner and placed on an IHW test disc.  In practice, previous pixel
compression on the L-SP images of Halley has resulted in files only slightly
larger than 50% the size of the original images.  Details of the algorithm are
given in the COMPRESS.TXT file of this directory, and some computer language
versions of the code are contained in the SOFTWARE directory.  A proposal for
handling compressed image data in FITS is given in the file FITSCOMP.TXT in
this directory.

     Due to the modest data transfer rate inherent in the CD-ROM technology,
the process of reading an image is quite time-consuming, even though data
compression has been applied.  For this reason, a set of subsampled or "browse
images" is also supplied as a data product on the compressed image discs.
These images are restricted to a maximum of 256 pixels on a side and preserve
the original sampling geometry.  All browse images have been rescaled from
their original pixel values to 8-bits; for the vast majority of the images the
original values were 10-bits or 2-bytes.  For a few images that were already
digitized by the observers and were not 2-bytes per pixel, the data were
rescaled from 4-bytes to 2-bytes before being compressed or subsampled.
Details on the construction of browse images can be found in the BROWSE.TXT
file of this directory, and the code used to make the images is given in the
MIDGET.FOR file of the SOFTWARE directory.

     For both the compressed (full-resolution) and browse images, placement
of the data on disc and the filenaming scheme have been based on the
time-of-observation chronology of the images (refer to Section 5 below).  For
the compressed images, the directory name is Cyymmmdd, where "yymmmdd"
indicates the start date for data on a particular compressed image CD-ROM.
For browse images the directory name on all the L-SP CD-ROMs is BROWSE.  The
average number of data files in a directory is somewhat less than 80 for all
the L-SP discs, but note that the FITS headers and PDS labels also appear in
each of the directories Cyymmmdd and BROWSE.  There is no subdirectory below
either of these data directories.

     Some LSPN observers submitted calibration data which reside on these
CD-ROMs as digital images.  Those data have been placed both as compressed and
browse images in the CALIB and BROWSE directories.   Calibration data include
step wedges, spot sensitometry, and standard star fields, and the details are
contained in the individual FITS headers associated with the calibration
datafiles.  The linkage between calibration data and specific images of
Comet Halley can be found in either the CALIB.TXT file in this directory, or
in the CALIB.IDX delimited index table in the INDEX directory.


4. SUPPLEMENTARY DIRECTORIES

     The volume and directory structures of this disc conform to the Level-1
standard specified by the ISO.  This format is widely accepted and used on a
variety of machines.  The AAREADME.TXT file in the root directory introduces
the user to the full extent of the archive and the contents of the individual
subdirectories.

     There are three directories (DOCUMENT, INDEX, SOFTWARE) on the CD-ROM
that contain supplementary files.  In addition, to correlate chronological
observations with physical location of the comet, detailed ephemeris
information has been included in a separate subdirectory EPHEM.  The table
(EPHEM.TAB) contains information at daily intervals, so an interpolation
routine is also provided; more information is in the text (EPHEM.TXT) file.

     The DOCUMENT subdirectory contains text files that give background to
this CD-ROM project and a general guide to its use.  Descriptions of the FITS
and PDS formats are in FITSINFO.TXT and PDSINFO.TXT.  IMAGUIDE.TXT is meant
as a general overview of the data products.

     Tables of useful INDEX information have been collected in various forms
in order to allow automated searching of the data.  For LSPN, the nearly full
set of FITS keywords contained in the headers is in a file named NETLARGE.IDX
which includes the filename but not the path.  A file, PATHTABL.IDX, gives the
location of the file separately.  We have attempted to make all index tables
transportable to relational DBMS by delimiting the tables and providing
structure and dBASE-compatible .DBF files.  The CALIB.IDX file has a separate
entry that relates each calibration to an image or range of images that can
be associated.  The files EPHEM, CDSTRUCT, and LSPNOBS are also provided as
delimited tables. Further information is contained in the file INDXINFO.TXT.

     The SOFTWARE subdirectory contains program code for a few utilities, in
this case, manipulation of images and the interpolation of ephemeris tables
for the comet.  The Previous Pixel algorithm used to compress the images is
described in the file FITSCOMP.TXT.  The code was developed by the L-SP
Discipline at Goddard Space Flight Center.  Pseudocode for this algorithm is
listed (PREVPIX.TXT) as well as the IDL procedure (PREVPIX.PRO) originally
used to test the compression on comet images.  Fortran code to compress and
subsample the data was composed for an IBM 3081 environment; the source code
and explanation files (BROWSE.TXT and COMPRESS.TXT) is in the appropriate
subdirectory under SOFTWARE.  For image decompression, both Fortran
and C code routines are provided; an MS-DOS program that accesses the split
FITS header and data is included as an example of executable programs.  An
interpolation program was developed by the Astrometry network group for the
ephemeris table.  The Fortran code, an equivalent C version, and executable
file for VAX/VMS and MS-DOS are provided.



5. FILENAMING CONVENTIONS

     The convention for naming files on the CD-ROM was adopted by the IHW
Lead Center to include a running number as a unique data qualifier for
chronological sorting.  The name for the net (LSPN in this case) has been
abbreviated and concatenated with the number to yield the appropriate
filename (e.g., LSPNxxxx).   For all calibration data, the filename is
LSPN4xxx to distinguish from the 3383 images archived for the Comet.

     The file extensions follow suggestions by the PDS (SPIDS v1.1) for
compressed (.IMQ) and subsampled (.IBG) data with detached label (.LBL).
Since the data was originally submitted as FITS files and now registered as
separate header and data files, a convention for header (.HDR) was adopted.
These extensions are listed below:

                       .HDR - FITS header
                       .IBG - subsampled (browse) image
                       .IMQ - compressed data
                       .LBL - detached (PDS) label

Note that in the ISO formatting process a version number (1 by default)
is attached to the filename.   In MS-DOS this information is ignored but
for complex systems such as VMS, it is a necessary piece of information.

     In splitting up the original FITS files, we have attempted to make the
data available to a larger community.  For transporting data to analysis
packages that demand FITS, the header and data files may be concatenated to
create a fully valid FITS byte stream, as the original structure of the FITS
headers and data have been preserved.



6. AVAILABLE SOFTWARE

     At the present, some executable software is included with the CD-ROM for
programs supporting the ephemeris and image decompression.  The data formats
are compatible with existing MS-DOS packages and subsequently we will provide
such software on floppy disk.  We have supplemented the CD-ROM release with an
accompanying printed manual for its use.



7. REFERENCES

     Martin, T., Martin, M., Davis, R.L., Mehlman, R., Braun, M., and Johnson,
M.: October 3, 1988, Standards for the Preparation and Interchange of Data
Sets, Version 1.1, JPL D-4683.

     "Information Processing -- Volume and File Structure of CD-ROM for
Information Interchange", Reference Number ISO 9660: 1988(E), Produced in
coordination with National Information Standards Organization, National Bureau
of Standards, Administration 101, Library E-106, Gaithersburg, MD.






      E. Grayzeck, Jr             M. Niedner, Jr.
      National Space Science      IHW Discipline Specialist
      Data Center                 for Large-Scale Phenomena

      Interferometrics, Inc       Laboratory for Astronomy
                                  and Solar Physics
      Code 633                    Code 684.1
      NASA/GSFC                   NASA/GSFC
      Greenbelt, MD               Greenbelt, MD
      20771                       20771




      A.Warnock III                D. Klinglesmith III
      IHW Team Member              IHW Team Member
      for Large-Scale Phenomena    for Large-Scale Phenomena

      ST Systems Corp              Laboratory for Astronomy
                                   and Solar Physics
      Code 681                     Code 684
      NASA/GSFC                    NASA/GSFC
      Greenbelt, MD                Greenbelt, MD
      20771                        20771